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The Examiner has rejected claims 1 to 16 as 
anticipated by Scott. The Applicants respectfully disagree 
that these claims are anticipated by Scott for reasons set 
out below, and request reconsideration of allowability of 
claims 1 to 16 as amended, and new claim 17. 

The present application describes a server for 
handling multimodal information, that is capable of 
supporting multi-modal terminals. Original claim 1 
defined a terminal interface for supporting one or more 
connections to user terminals ...for passing information in 
at least one of the modal forms . Claim 1 has been amended 
to more clearly define the invention by specifying that the 
terminal interface provides support for multimodal 
terminals, i.e. ... for supporting one or more connections 
from the server to user terminals comprising at least one 
multimodal user terminal, and that the service controller 
is operable to interact with user terminals for controlling 
input or output of the information on the terminal 
interface and the internet interface according to the modal 
form of the input and output - 

Dependant claim 2 has been amended 
correspondingly, since support for multi-modal terminals is 
now defined more explicitly in claim 1. Claims 3 to 5 and 
7 have been amended for consistency in terminology. 

With respect to the Examiners rejection of claims 
9 as being a substantial duplicate of claim 8, the 
applicants note that the Examiner is correct that a 
duplicate claim was inadvertently included, and claim 9 has 
now been cancelled. 

Independent claims for other aspects of the 
invention, i.e. claim 12 for a server, claim 14 for a 
method of using a server, claim 15 for a method of using a 
multimodal service provided by a server, and claim 16 for 
software for carrying out the method of using a server, 
have been amended similarly to claim 1 amendments, to 
clarify that the server supports multimodal terminals and 
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different modal forms of input and output. Claim 13 
dependent on claim 12 has been amended for consistency in 
terminology with amended claim 12 . 

Thus the servers and methods described and claimed 
in the present application are capable of supporting 
multimodal terminals, e.g. a personal computer or a phone 
providing a multimodal interface, for exsunple as described 
in copending application 08/992,630. filed concurrently 
with this application, referenced on page 1 of the present 
application. Advantages of having multi -modal capability 
or modal sensitivity in the server rather than only in the 
user's terminal are noted on page 3, lines 24 to page 5, 
line 14 . 



Claims rejections 

The Examiner has rejected claims 1 to 16 as 
anticipated by Scott et al. (US 6,101,473). 

The Examiner argues with respect to claims 1, 7 
and 11, that Scott discloses a system for using speech 
recognition to access the internet comprising the following 
features: in Fig 1 speech web browser 7 for supporting one 
or more connections on the internet; speech web server for 
supporting one or more connections from server to user 15 
depicted in figure 5 and for passing information to the 
user 15; the speech web server for controlling input or 
output of the information in the server and for processing 
the information. 

The Examiner argues with respect to claims 2 and 
3, speech web server is operable to interact with user 15 
having a computer 16 and a telephone 19 and to select which 
modal forms to use according to the command received from 
the user. 

The Examiner argues with respect to claim 4 that 
speech web server is operable to receive inputs from user 
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15 and to determine the intention of the user based on 
inputs . 

The applicants respectfully disagree that the 
invention described in the present invention is anticipated 
by Scott for the following reasons. 



multimodal terminal/ nor of controlling inputs and outputs 
in different modal forms, as described in the present 
application, and now more clearly defined in claim 1. 

Scott provides a system fo r linking conventional 
components to handle graphical input using a conventional 
local web browser 1, such as Netscape, linked to a speech/ 
web browser and speech server residing on speech/ web 
server 8. As acknowledged in col. 3 line 12, the general 
purpose of Scott is to provide a method of linking known 
single mode systems, i.e. linking a remote speech 
recognition device operating over the telephone network to 
any conventional web browser operating over the internet, 
to provide an immediate solution for speech enabling the 
net or net enabling the telephone. Scott provides an 
interface to a conventional graphical interface, or to a 
telephone terminal capable of handling voice only and 
allows tandem operation, but Scott does not provide a 
truly multi-modal interface nor support for one or more 
multi-modal terminals (see col 3, lines 22 to 24) where 
support for multimodal input and output is integrated - 
Scott fails to provide a service controller which operable 
to interact with a multi-modal terminal to select which 
modal form or forms to use, and for controlling input or 
output of the information on the terminal interface and the 
internet interface according to the modal form of the 
input and output as described in the present application 
and defined in amended claim 1. 



The server of Scott is not capable of supporting a 
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Scott discloses a system which is a combination 



three major components - speech web browser 7, speech 
server 9, speech server with telephony functions, and ASTP 
controller software 4 and 6 - the latter being described as 
the the heart of the system. Speech server 9 is described 
as a standard off the shelf web browser with a plug in 
(e.g. col 4, line 64) and speech/ web browser 7 is driven 
by speech only (col S line 2) . Disadvantages of a system 
having separate graphical and speech interfaces, such as in 
Scott, are described on page 2, line 28 to page 3 line 7 of 
the present applications, which highlights issues with 
synchronization of separate interfaces for different modes 
of input and output. Specifically, speech web interface 
of Scott is a speech/audio only interface while the 
conventional browser is a graphics interface. Neither does 
Scott provide a multi-modal interface capable of accepting 
input and providing output in more than one modal form as 
described in the present application. 



defined in amended claim 1 distinguishes patentably over 
Scott. If claim 1 is found to be allowable, dependent 
claims 2 to 11 which define further limitations thereon 
should also be patentable. New independent claim 17 
defines a server including the limitations of claim 1 in 
combination with the limitation of claims 2, 3, and 4, and 
thus in particular provide support for... receiving inputs 
simultaneously in different modal forms . 



aspects of the invention have been amended correspondingly 
with the amendments made to claim 1, and if claim 1 is 
found to be allowable, these claims should also be 
allowable over Scott for reasons stated above. Claim 13 is 
dependent on claim 12 and add further limitations thereto. 



Thus the Applicants believe that the server 



Independent claims 12, 14, 15, 16 for other 
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Thus the Applicants respectfully request 



reconsideration of amended claims 1 to 16, and new claim 17 
in light of the arguments present above and with a view to 
allowance . 



AdeW : go * d 

C/O NORTEL NETWORKS l/IMITED. 
Intellectual Property Law Group, 
Box 3511, Station C 
Ottawa, Ontario, Canada, KlY 4H7 

Phone: (613) 768-3020 

Date: March 21, 2001 



Yours very truly. 



H,A. PASTERNACK et al . 




Angela C. de Wilton, 
Patent Agent 

Registration No. 35,763^ 
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Marked up copy of original claims showing amendments 



1. A server for hsmdling , information which is in 
different modal forms suitable for more than one mode of 
user interface, the server comprising: 

an internet interface for supporting one or more 
connections on the internet; 

a terminal interface for supporting one or more 
connections from the server to user terminals , at least one 
terminal being a multi -modal user terminal , and for passing 
information in at least one of the modal forms; and 

a service controller being operable to interact with 
user terminals for controlling input or output of the 
information on the terminal interface and the internet 
interface, and for processing the information received from 
or sent to either interface, according to ito the m odal 
form of the input and output . 

2. The server of claim 1, the service controller 
being operable to intGract with a ujor having with a multi- 
modal user terminal- — attd to select which modal form or 
forms to use . 

3. The server of claim 2, the service controller 
being arranged to malco tho DclGGtion select which modal form 
or forms to use— according to the content of the 
information, and the context of the interaction. 

4. The server of claim 1, the service controller 
being operable to receive inputs in different mode s modal 
forms simultaineously from the same user, to resolve any 
conflicts, and determine an intention of the user based on 
the inputs . 
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5. The server of claim 1, the terminal interface 

being arranged to recognize recognize speech as an input. 

6- The server of claim 1, the terminal interface 

being arranged to generate audio as an output mode, 

7. The server of claim 1, the service controller 
being arranged to conduct a dialogue with the user in the 
form of a sequence of interactions. 

8. The server of claim 1, further comprising means 
for initiating a connection to the user's terminal. 

5- The Dcrvcr of olaim 1, — further oomprioing mcanQ 

for initiating a conncGtion to the UDor'n terminal. 

10- The server of claim 1, further comprising a link 

to a telephone network, and a call processor for making and 
receiving telephone calls on the telephone network, 

11. The server of claim 1, comprising a set of 
servlets arranged to communicate with each other by event 
driven messages. 

12 , A server for interfacing between telephone calls 
and the internet, and comprising: 

a terminal interface f or supporting one or more 
connections from the server to user terminals, the terminal 
interface providing a telephony interface for receiving or 
making a telephone call, and arranged to interact with a 
user on the call by rPGonniaina recognizing speech or 
generating audio signals; 

an internet interface for receiving or outputting 
information from or to other parts of the internet; and 

a service controller for controlling interaction 
between the terminal interface, telephony interface and the 
internet interface^- 
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the service controller being opera hig^ i-n inceract with 
a user having a terminal capable of handling t elephony or 
internet communications, to select which modal form or 
forms to use for controlling input or outpu t of the 
information on the terminal interface and the internet 
interface, according to the modal form of th e input and 
output . 

13 The server of claim 12, wherein the controller 

being is operable to interact ^i^ith a user having a 
terminal oapablc of handling telephony or internet 
communicat iono , — the Gontrollor further being operable to 
select which of -febe& etelephonv or internet communications 
to use when sending information to the terminal, and to 
convert the information to suit the selected type of 
communication . 

14. A method of using a server to handle information 
in different modal forms suitable for more than one mode of 
user interface, and comprising the steps of: 

supporting one or more connections on the internet; 

supporting one or more connections from the server to 
the user terminals , at least one terminal being a multi- 
modal user terminal : 

passing information in different modal forms between 
the user and the server; 

controlling input or output of the information on the 
terminal and internet interfaces; and 

processing the information received from or sent to 
either interface, according to - its the m odal for m of the 
input and output . 

15. A method of using a multi-modal service provided 
by a server on the internet, the server having an internet 
interface for supporting one or more connections on the 
internet, a terminal interface for supporting a connection 
to a user of the service, and for passing information in « 
least one of the different m odal forms; and a service 
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controller for controlling input or output of information 

on the terminal interface and the internet interface, and 

for processing the information received from or sent to 

either interface, according to ita the modal form, the 

method comprising the steps of: 

providing input to the terminal interface of the 

servers- 
engaging in a dialogue with the server to cause the 

server to process the information according to its the 

modal form of the information; and 

receiving a response from the terminal interface of 

the server, according to the result of the information 

processing, 

16 . Software on a computer readable medium for 

carrying out a method of using a server to handle 
information in different modal forms suitable for more than 
one mode of user interface, the method comprising the steps 
of: 

supporting one or more connections on the internet; 
supporting one or more connections from the server to 

the user terminals , including multi-modal user terminals ; 

passing information in different modal forms between a 

-fet^^user terminal and the servers- 
controlling input or output of the information on the 

terminal and internet intcrf acoa connections ; and 

processing the information received from or sent to 

either intorf acc connection . according to ifrs - the m odal form 

of the input and output . 
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